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Hands-On  Scorer  Training  Materials 

Overview  of  Hands-On  Training  and  Test  Administration 
Hands-On  Scorer  Orientation  Handout 
Scorer  Orientation  Briefing 


OVERVIEW  OF 

HANDS-ON  TRAINING  AND  TEST  ADMINISTRATION 


GUIDANCE  FOR  HOM 


A.  Pre- training 

1.  Familiarize  yourself  with  the  station  setup  and  each  individual  test.  Even 
If  you  are  familiar  with  the  tasks,  re-read  them  before  you  go  to  scorer 
training. 

2.  Visit  the  hands-on  site  and  decide  where  each  station  will  be  located  and 
where  control  points  for  examinees  will  be. 

3.  Contact  the  units  that  are  providing  the  equipment  and  the  scorers  and  make 
sure  they  know  when  and  where  to  be.  Do  this  the  day  before  scorer  training 
even  though  prior  coordination  has  been  made. 

B.  Scorer  Orientation 

1.  For  the  first  hour  or  two  of  scorer  orientation  meet  all  the  scorers  in  a 
comfortable  setting— a  classroom  if  possible.  Pass  out  the  Hands-On  Scorer 
Roster  and  have  the  scorers  fill  out  the  information  (except  the  Station 
Assigned).  Issue  a  copy  of  the  Hands-On  Scorers'  Orientation  Outline 
hand-out  to  each  scorer  and  go  through  the  Scorer  Orientation  Briefing. 
Outline  the  activities  to  follow  in  Scorer  Training  and  during  Test 
Administration.  Ascertain  that  the  scorers  must  be  present  during  the 
entire  test  period  (no  TDY,  schooling,  duty  or  personal  appointments.) 

2.  Pass  out  a  copy  of  the  Station  List/Equipment  List  to  each  of  the  scorers. 
Explain  any  tests  that  are  modified  from  their  task  titles.  Tell  the 
scorers  that  you  want  them  to  choose  which  station  they  will  be  scoring— the 
tasks  they  are  most  famil iar  with  and  will  be  the  most  comfortable  scoring. 
Then  start  with  the  first  station  ar.d  make  the  assignments.  If  no  one  wants 
a  station,  or  two  people  want  the  same  station,  you  make  the  judgment  based 
on  your  judgment  as  to  who  is  most  familiar  with  the  tasks. 

3.  Write  down  the  assignments  on  the  Scorer  Roster.  If  you  have  more  scorers 
than  stations,  keep  them.  Assign  them  to  the  most  difficult  stations  (in 
your  judgment).  You  may  modify  your  station  assignments  during  or  after 
scorer  training. 

4.  Pass  out  copies  of  the  tests  assigned  to  each  scorer.  Go  over  the  general 
organization  of  the  Scorer's  Instructions  and  Scoresheet.  Pass  out  a  copy 
of  the  Station  Cards  (carried  by  each  soldier  tested)  and  explain  how  the 
scorers  will  obtain  the  Scorer  ID  Number  and  initial  when  a  soldier 
completes  the  station.  Go  through  the  heading  of  the  scoresheet  and  stress 
the  importance  of  accurately  filling  in  the  scorer's  name,  soldier's  name, 
and  ID  number. 
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5.  Before  leaving  the  classroom  setting  each  scorer  must  read  the  Scorer's 
Instructions  and  Scoresheets  for  his  tests.  Allow  enough  time  for  this  and 
for  slow  readers.  The  importance  of  reading  their  materials  deserves 
emphasis.  Many  later  problems  arise  because  scorers  have  not  thoroughly 
read  the  printed  materials  provided. 

C.  Scorer  Training 

1.  Move  to  the  hands-on  location  and  coordinate  distribution  of  equipment 
to  each  station.  Have  each  scorer  set  up  his  test  station  and 
equipment  in  accordance  with  the  Scorer  Instructions. 

2.  Have  scorers  take  turns  acting  as  examinees  on  each  station.  The 
actual  scorer  should  administer  and  score  the  test  exactly  as  required 
for  actual  testing. 

3.  You  must  observe  a  minimum  of  one  practice  administration  of  each 
station.  Watch  the  entire  administration  from  restoring  equipment 
conditions  to  filling  out  the  scoresheet.  Critique  the  scorer,  both 
good  and  weak  points,  after  the  administration.  Be  specific  in  your 
debrief.  Don't  ask  the  scorer  if  he  has  any  problems--they  will 
usually  reply  that  they  don't.  It's  up  to  you  to  identify  problems. 

Always  give  specific  solutions  to  problems  that  you  identify. 

4.  For  each  task,  be  prepared  to  introduce  common  or  likely  errors  via  the 
"examinee"  to  insure  the  scorer  reacts  properly  to  unusual  situations. 

5.  Some  scorers  will  need  to  practice  all  or  certain  parts  of  th  scoring 
process  several  times.  How  many  times  depends  on  the  scorer  and  the 
task.  A  minimum  of  two  run  throughs  is  usually  required;  some  scorers 
will  require  many  more.  But  do  not  have  a  scorer  continue  to  practice 
on  his  own  just  to  have  him  doing  something  while  you  check  out  the 
rest  of  the  practices.  The  most  worthwhile  practices  are  the  ones  you 
observe. 

6.  Take  note  of  scorer  weaknesses  as  a  basis  for  later  monitoring.  Where 
errors  are  observed  try  to  relate  the  error  to  either  the  scoresheet  or 
Scorer's  Instructions  as  a  foundation  for  correction.  This  will  give 
the  scorer  a  specific  reference  for  future  behavior.  Stress  to  scorers 
that  they  are  not  to  modify  or  deviate  from  the  Instructions  and 
measures  they  have  been  given.  Make  sure  that  they  know  to  summon  you 
during  test  administration  if  any  situation  arises  that  they  are  not 
sure  of  and  not  to  react  on  what  they  think  should  be  done. 

7.  Before  dismissing  scorers  from  scorer  training,  debrief  them  in  a  group 
if  possible.  Reiterate  the  fundamentals  of  scoring  and  make  sure  they 
know  their  commitments  for  the  test  period.  Keep  the  debriefing  upbeat 
and  positive. 
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0.  Test  Administration 

1.  Assemble  the  scorers  just  prior  to  the  first  session's  administration 
and  stress  again  the  requirement  to  call  you  for  any  unexpected 
developments.  Check  each  station  and  equipment  setup  prior  to  the 
arrival  of  troops. 

2.  During  the  first  two  hours  of  the  first  session,  visit  each  station  and 
observe  one  administration  of  each  complete  station.  Concentrate  initially 
on  those  weak  areas  and/or  scorers  you  noted  during  training.  But  don't 
ignore  the  other  stations.  Scorers  will  react  differently  when  actual 
soldiers  are  Involved  and  new  problems  will  arise.  Be  alert  for  troop 
control . 

3.  Check  scoresheets  frequently,  especially  immediately  after  observing  a 
performance  to  see  if  your  observation  corresponds  with  the  scoresheet. 

4.  Make  corrections,  offer  advice  to  scorers  as  necessary  but  always  away  from 
the  troops  being  tested.  Offer  encouragement  and  sincere  commendations  as 
well . 

5.  Continue  to  circulate  to  stations  throughout  the  test  sessions.  Be  alert 
for  signs  of  scorer  fatigue,  indifference  or  sloppiness  especially  toward 
the  end  of  test  sessions. 

6.  Consistent  with  test  requirements,  insure  that  scorers  get  short  breaks  to 
have  a  cigarette,  refreshment  or  go  to  the  latrine.  Some  tests  are  more 
fatiguing  than  others.  Try  to  spare  these  scorers  for  slightly  longer 
periods  by  having  someone  take  the  station  for  a  few  iterations,  if 
personnel  resources  allow. 

7.  Don't  assume  that  things  are  going  all  right  just  because  soldiers  appear  to 
be  moving  through  a  station.  Check  everything— and  more  than  once.  If  you 
don't  take  anything  for  granted,  neither  will  your  scorers. 

8.  When  multiple  test  administrations  are  used,  analyze  the  scoresheets  between 
sessions.  Be  alert  for  a  pattern  of  all  GO  performance,  "impossible" 
performance,  inconsistent  rating,  incomplete  rating.  Make  corrections 
before  the  next  test  session  and  direct  your  monitoring  towards  those 
stations. 
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(Handout) 


OVERALL  OBJECTIVE  OF  THE  PROJECT 

•  To  look  at  how  well  entry  tests  (like  ASVAB)  predict  actual  job  performance. 

•  Three  parts  to  the  project: 

1.  Select,  develop  and  try-out  entry  tests. 

2.  Develop  and  try-out  job  performance  measures. 

3.  Look  at  the  relationship  between  the  entry  test  and  job 
performance  measures. 

OBJECTIVE  OF  THIS  PHASE 

•  Administer  job  performance  measures 

1.  Written  tests 

2.  Ratings  by  supervisors  and  peers 

3.  Hands-on  measures 

•  Administer  "entry"  tests 

OBJECTIVE  OF  ADMINISTRATION  OF  HANDS-ON  MEASURES 

•  Administer  tests  in  a  standardized  manner. 

1.  Equipment  conditions  and  set-up. 

2.  Scorers'  manner  and  method  of  scoring: 

a.  Treat  all  soldiers  professionally. 

b.  Present  instructions  as  written. 

c.  Score  every  performance  measure. 

•  Test,  don't  train. 

1.  Unless  specific  instructions  say  so,  do  not  tell  soldiers 
how  to  do  any  step. 

2.  Instruct  soldiers:  "Do  the  best  you  can." 

3.  If  a  soldier  cannot  perform  a  step  after  trying: 

a.  Score  the  step  NO-GO. 

b.  Tell  the  soldier:  Go  on  with  the  rest  of  the  task. 

c.  If  a  soldier  cannot  continue,  write  "stopped"  under 
COMMENTS  and  score  the  rest  of  the  measures  NO-GO. 

4.  Provide  NO  Feedback: 

a.  Be  careful  what  you  say. 

b.  Watch  your  expressions. 

•  Good  scorers  are  the  most  important  part  of  hands-on  testing. 

K.2  Hands-On  Scorer  Orientation  Handout 


K-5 


SCORER  ORIENTATION  BRIEFING 


The  overall  purpose  of  this  project  is  to  develop  a  comprehensive  selection  and 
classification  system  for  the  Army.  The  major  part  of  the  work  is  to  look  at  the 
relationship  between  how  well  soldiers  do  on  the  entry  tests  (like  ASVAB)  and  how 
well  they  do  on  the  job.  At  this  post  SL1  soldiers  in  nine  MOS  will  take  four  kinds 
of  measures;  one  kind  is  hands-on  measures.  For  these  hands-on  measures  soldiers 
will  perform  about  15  of  the  tasks  required  by  their  job. 

The  success  of  the  hands-on  tests  depends  on  you,  the  scorers  who  will  rate  each 
soldier's  performance.  We  will  ultimately  test  over  1,000  soldiers  at  many 
different  locations.  About  30  different  scorers  will  evaluate  some  soldiers  on  the 
same  task.  For  the  results  to  mean  anything,  things  must  be  as  much  the  same  as  we 
can  make  them— regardless  of  where  the  installation  is,  or  who  is  doing  the  scoring. 
For  that  to  happen  all  your  actions  as  a  scorer  must  be  guided  by  two  principles: 

•  Be  sure  the  test  conditions  are  the  same  for  every  soldier. 

•  Apply  the  standard  evenly  to  every  soldier. 

Let's  consider  the  first  principle.  The  test  conditions  are  all  the  factors 
that  determine  the  difficulty  of  a  task.  For  example,  think  of  the  test  conditions 
for  a  task  like  changing  the  tire  on  a  quarter  ton  truck.  The  test  is  more 
difficult  outside  than  inside  the  motor  pool.  If  the  test  is  outside,  it  is  harder 
on  a  hill  than  on  hardstand.  The  test  is  also  harder  if  you  do  not  have  the  right 
tool  or  if  the  tools  do  not  work  correctly.  It  is  also  harder  to  change  the  tire  by 
yourself  than  if  someone  helps  you. 

The  people  who  wrote  the  tests  have  described  the  general  requirements,  please 
do  not  deviate  from  their  guidance  without  working  through  (the  HOM).  There  may 
also  be  times  when  you  will  find  some  variations  in  the  equipment  that  the  developer 
did  not  anticipate.  For  example  we  may  not  have  requested  all  the  necessary  tools 
or  some  tool  we  did  request  may  not  work  correctly.  In  all  such  cases  notify  (the 
HOM)  Immediately.  The  results  will  not  mean  anything  if  the  conditions  are  not  the 
same. 

The  results  also  will  not  mean  anything  if  every  scorer  does  not  apply  the 
standard  evenly  to  every  soldier.  The  standard  describes  how  well  a  soldier  must  do 
a  task  to  pass  the  test.  The  standard  is  listed  as  performance  measures.  For  tests 
where  you  must  watch  a  soldier  do  the  task,  the  performance  measures  tell  you  what 
steps  must  be  done,  how  to  tell  if  the  step  is  done  right,  if  the  steps  must  be  done 
in  a  certain  sequence,  and  whether  there  is  a  time  limit.  For  tests  where  you  must 
check  the  soldier's  work  after  the  test  (such  as  filling  out  a  form),  the 
performance  measures  tell  you  what  points  to  check  and  the  time  limit.  In  both 
cases,  every  performance  measure  is  important  and  should  be  applied  in  the  same  way 
to  every  soldier. 

Now  let's  consider  some  implications  of  these  two  principles.  First  you  must 
maintain  the  test  conditions  so  that  soldiers  at  the  end  of  the  day  do  not  get  help 
from  the  equipment  or  station.  You  may  need  to  clean  equipment  periodically  or 
clean  up  around  the  station  to  get  rid  of  clues.  Also  you  may  need  to  do  some  set 
up  steps  before  each  soldier.  If  you  do,  the  cover  sheet  will  tell  you  what  steps 
are  needed.  Keep  that  sheet  with  you. 
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Your  first  contact  with  the  soldiers  you  test  will  be  when  you  read  the 
instructions.  Be  sure  you  read  them  precisely  as  they  are  written  on  the 
scoresheet. 

When  you  read  the  instructions,  and  in  all  your  contact  with  soldiers  being 
tested,  your  facial  expression,  voice  inflection,  and  posture  should  be  the  same. 

You  may  hope  people  you  like  do  well  and  people  you  do  not  like  do  poorly,  but  you 
must  treat  everyone  the  same.  Your  facial  expression,  voice,  and  posture  must  not 
threaten  soldiers  you  test.  Your  demeanor  should  be  objective,  professional,  and 
non-threatening. 

In  most  cases  you  will  score  the  test  by  watching  the  soldier  perform  the  task. 
Try  to  mark  the  performance  measures  as  the  soldier  performs  the  task.  If  you  wait 
until  the  end,  you  may  forget  what  happens.  If  you  only  mark  the  mistakes,  soldiers 
might  get  too  much  information  on  how  well  they  are  doing.  If  something  happens 
that  you  are  not  sure  how  to  score,  jot  down  a  reminder  of  what  happened  under 
COMMENTS  and  get  with  (the  HOM)  before  you  test  the  next  person. 

When  you  evaluate  performance,  rate  each  measure,  do  not  add  check-points  to  the 
scoresheet  and  do  not  ignore  any  performance  measure. 

There  is  one  tendency  that  is  especially  troublesome.  If  you  are  like  most  good 
NCOs,  you  will  want  to  train  soldiers  on  the  task.  If  you  see  someone  make  a 
mistake,  every  fiber  of  your  NCO  body  will  ache  to  correct  the  mistake.  It  is 
important,  though,  that  you  hold  back.  Part  of  the  information  we  must  collect 
tells  how  well  people  do  on  different  kinds  of  tests  for  the  same  task  and  how  their 
score  changes  from  time  to  time.  If  there  are  differences  between  the  types  of 
tests  or  between  the  times  soldiers  are  tested,  we  will  conclude  that  something  is  . 
wrong  with  the  test.  To  correct  the  problems,  we  need  to  be  sure  that  the  cause  of 
changes  was  not  that  you  told  them  the  right  way.  So  the  rule  is— Provide  No 
Feedback. 

Another  reason  for  not  training  soldiers  is  the  first  principle— every  soldier 
must  be  treated  the  same  not  only  by  you,  but  by  every  scorer  in  CONUS  and  Europe. 
For  the  same  reason  you  must  be  careful  how  you  react  when  soldiers  have  problems 
during  the  test.  Unless  (the  HOM)  or  your  test  Instructions  tell  you  differently, 
do  not  tell  soldiers  how  to  do  any  step.  If  the  soldier  gets  stuck  or  says  he  or 
she  does  not  know  what  to  do  next,  say,  "Do  the  best  you  can."  Allow  about  one 
minute  and  say  "Go  on  with  the  rest  of  the  task."  If  the  soldier  cannot  go  on, 
write  "stopped"  under  COMMENTS  and  mark  the  rest  of  the  measures  in  this  section 
NO-GO.  If  you  think  a  soldier  is  pretending  not  to  be  able  to  do  anything,  write 
"no  effort"  under  COMMENTS  and  send  the  soldier  to  the  control  NCO. 

We  conclude  with  the  same  point  we  started  with.  You  are  the  most  important 
person  for  us  to  get  information  on  these  tests  that  means  anything.  A  lot  of 
things  can  go  wrong  in  this  kind  of  project,  but  we  must  know  that  the  scorers  did 
their  job. 
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Knowledge  Test  Monitor  Instructions 

L.l  Sample  Monitor  Instructions  (Batch  A) 

L.2  Monitor  Instructions  (Batch  B) 


L-l 


MONITOR  INSTRUCTIONS 
13B  K- Tests 


1.  In  the  Initial  test  session,  pass  out  7  copies  of  Test  Booklet  A1  and  7 
copies  of  the  white  answer  sheet  and  8  copies  of  Test  Booklet  A2  and  8  copies  of 
the  green  answer  sheet.  Make  sure  that  every  one  who  has  a  green  answer  sheet 
has  a  green  cover  sheet  on  their  test  booklet  and  the  same  for  white.  The  next 
test  session,  use  8  copies  of  white;  7  copies  of  green  and  so  on  throughout  the 
test  session  to  keep  the  number  of  each  test  version  used  even. 

2.  Use  the  following  test  schedule  sequence. 


Test  Session  1: 

A 

- 

1/2 

Test  Session  6  (b): 

B 

- 

1/2 

B 

- 

1/2 

C 

- 

1/2 

C 

- 

1/2 

D 

or 

E 

- 

1/2 

D 

or 

E 

- 

1/2 

A 

- 

1/2 

Test  Session  2: 

B 

1/2 

Test  Session  7: 

C 

1/2 

C 

- 

1/2 

D 

or 

E 

- 

1/2 

D 

or 

E 

- 

1/2 

A 

- 

1/2 

A 

- 

1/2 

B 

- 

1/2 

Test  Session  3  (a): 

C 

_ 

1/2 

Test  Session  8: 

D 

or 

E 

_ 

1/2 

D 

or 

E 

• 

1/2  • 

A 

- 

1/2 

A 

- 

1/2 

B 

- 

1/2 

B 

- 

1/2 

C 

- 

1/2 

Test  Session  4  (a): 

0 

or 

E 

_ 

1/2 

Test  Session  9: 

A 

_ 

1/2 

A 

- 

1/2 

B 

- 

1/2 

B 

- 

1/2 

C 

- 

1/2 

C 

- 

1/2 

D 

or 

E 

- 

1/2 

Test  Session  5  (b): 

A 

1/2 

Test  Session  10: 

B 

_ 

1/2 

B 

- 

1/2 

C 

- 

1/2 

C 

- 

1/2 

D 

or 

E 

- 

1/2 

0 

or 

E 

- 

1/2 

A 

- 

1/2 

(a)  Test  Sessions  3 

and 

1  4 

are 

conducted  simultaneously. 

(b)  Test  Sessions  5 

and 

I  6 

are 

conducted 

simultaneously. 

3.  Separate  the  M109  and  M110  soldiers  at  the  beginning  of  each  session.  All 
soldiers  will  take  test  groups  A,  B,  and  C  1  or  2.  Only  M109  soldiers  will  take 
Group  D  1/2  and  only  MHO  soldiers  will  take  Group  E  1/2. 

4.  Make  sure  each  soldier  has  a  sharpened  pencil  with  an  eraser.  Sharp 
pencils  are  a  must  for  Test  Booklet  B  1/2  for  the  "Azimuth  with  Protractor" 
test. 
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5.  Read  the  following  Instructions  to  the  soldiers: 

DURING  THE  NEXT  FOUR  HOURS  YOU  WILL  BE  COMPLETING  WRITTEN  TESTS  ON  TASKS  IN 
YOUR  MOS.  THE  TEST  IS  DIVIDED  INTO  4  BOOKLETS  AND  COVERS  ALTOGETHER  30  TASKS. 
YOUR  FIRST  BOOKLET  IS  ON  YOUR  DESK.  YOU  WILL  NOTICE  THAT  THE  BOOKLET  HAS  A 
COVER  SHEET  WITH  THE  LETTER  (A,B,C,D,E)  ON  IT  FOLLOWED  BY  EITHER  THE  NUMBER  ONE 
OR  THE  NUMBER  TWO.  M109  SOLDIERS  WILL  TAKE  A,B,C,D  BOOKLETS  WHILE  MHO  SOLDIERS 
WILL  TAKE  A,B,C,E  BOOKLETS.  YOU  WILL  ALSO  NOTICE  THAT  YOUR  COVER  SHEET  ON  YOUR 
TEST  BOOKLET  IS  EITHER  WHITE  OR  GREEN.  NOW  PICK  UP  YOUR  ANSWER  SHEET.  IF  YOUR 
TEST  BOOKLET  HAS  A  WHITE  COVER  SHEET,  THEN  YOUR  ANSWER  SHEET  MUST  ALSO  BE  WHITE. 
IF  YOUR  TEST  BOOKLET  HAS  A  GREEN  COVER  SHEET  THEN  YOUR  ANSWER  SHEET  MUST  ALSO  BE 
GREEN.  IS  THERE  ANYONE  WHOSE  SHEETS  DO  NOT  MATCH?  (Pause-Check)  THE  FIRST 
THING  I  WANT  YOU  TO  DO  IS  TO  WRITE  YOUR  NAME  AND  SOCIAL  SECURITY  NUMBER  ON  THE 
SIDE  OF  THE  ANSWER  SHEET.  ALSO  THE  DATE.  DO  THAT  NOW.  (Pause- Check)  ABOVE 
THE  DATE  WRITE  IN  THE  LETTER  AM  (PM).  (Pause) 

NOW  LOOK  AT  YOUR  ANSWER  SHEET.  NOTICE  THAT  YOUR  ANSWER  SHEET  IS  DIVIDED 
INTO  BLOCKS  A  1  OR  2  AND  B  1  OR  2  ON  THE  FRONT  AND  C  1  OR  2  ON  THE  BACK.  ALL 
M109  SOLDIERS  SHOULD  HAVE  D  1  OR  2  AND  M110  SOLDIERS  SHOULD  HAVE  El  OR  2.  MAKE 
SURE  THE  BLOCK  YOU  ARE  WORKING  IN  MATCHES  THE  LETTER  AND  NUMBER  ON  YOUR  TEST 
BOOKLET.  THE  NAMES  OF  THE  TASKS  ARE  ON  THE  ANSWER  SHEETS  TO  HELP  YOU. 

WHEN  YOU  TAKE  THE  TEST,  READ  THE  ITEM  IN  THE  TEST  BOOKLET  AND  SELECT  THE 
ANSWER  YOU  CHOOSE.  THEN  GO  TO  THE  ANSWER  SHEET  AND  DRAW  A  LINE  OR  AN  X  THROUGH 
THAT  LETTER  ON  THE  ANSWER  SHEET.  DOES  EVERYONE  UNDERSTAND?  ANY  QUESTIONS? 

(Use  an  example  to  Illustrate,  if  necessary)  DO  NOT,  I  REPEAT,  DO  NOT  MARK  YOUR 
ANSWERS  IN  YOUR  BOOKLET  OR  MAKE  ANY  OTHER  MARKS  IN  YOUR  BOOKLET.  ONE  SHEET  OF 
BLANK  PAPER  IS  PROVIDED  FOR  YOU  TO  MAKE  ANY  COMPUTATIONS  ON. 

MARK  ONLY  ONE  ANSWER  FOR  EACH  QUESTION.  IF  YOU  WISH  TO  CHANGE  YOUR  ANSWER, 
ERASE  YOUR  OLD  MARKS  COMPLETELY.  ANSWER  EVERY  ITEM  -  THERE  IS  NO  PENALTY  FOR 
WRONG  ANSWERS. 

WHEN  YOU  HAVE  COMPLETED  THE  BOOKLET,  CHECK  YOUR  WORK.  THEN  YOU  MAY  TAKE  A 
BREAK.  LEAVE  ALL  YOUR  MATERIALS  ON  YOUR  DESK.  I  WILL  CALL  A  MANDATORY  10 
MINUTE  BREAK  IN  50  MINUTES  AND  ISSUE  YOUR  SECOND  TEST  BOOKLET. 

DO  NOT  HURRY  YOUR  WORK.  MOST  OF  YOU  WILL  FINISH  LONG  BEFORE  THE  50  MINUTES 
ARE  UP. 

FINALLY,  I  REMIND  YOU  THAT  YOU  MUST  DO  YOUR  OWN  WORK  AND  THERE  WILL  BE  NO 
TALKING  ALLOWED  DURING  THE  TEST  AND  DO  NOT  TALK  ABOUT  THE  TEST  DURING  YOUR 
BREAK.  ARE  THERE  ANY  QUESTIONS? 

6.  Monitor  during  the  test  to  insure  that  each  soldier  is  working  within  the 
correct  block.  This  is  especially  important  if  the  group  starts  other  than  on 
the  A  block. 
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7.  If  any  soldier  needs  to  leave  the  immediate  break  area,  insure  they  have 
your  permission.  If  all  soldiers  complete  a  booklet  before  the  50  minutes  are 
up,  begin  the  next  test  session  10  minutes  after  the  last  soldier  finishes.  If 
there  is  a  soldier  who  is  unusually  slow,  you  may  allow  her  to  work  through  the 
break  if  they  choose  to  do  so.  But  start  that  soldier  on  the  next  booklet  along 
with  the  others. 

8.  When  soldiers  are  taking  Test  Booklet  B  1/2,  for  the  task  "Measure  Azimuth 
With  A  Protractor,"  you  must  pass  out  1  protractor,  6TA  6-5-2  and  1  copy  of 

Ft.  Knox  Map  Extract.  Soldiers  should  draw  lines  on  the  map,  therefore,  each 
soldier  tested  will  need  a  fresh  map.  Insure  protractors  are  collected  when 
test  is  completed. 

Test  booklet  C  1/2,  task  "Prepare  DA  Form  2404,"  requires  the  7M  Extract  on 
the  1/4  ton  truck*.  Remind  the  soldiers  that  they  must  use  the  extract  to  answer 
parts  of  the  test. 

9.  When  testing  Booklet  B  1/2  for  task  "Prepare  Ammunition,"  call  the  soldiers 
attention  to  the  ammunition  charts  in  the  booklet.  He  will  use  these  to 
identify  ammunition  and  use  the  corresponding  letter  for  each  item  to  record  his 
responses  on  the  answer  sheet. 

10.  When  testing  Booklet  D  1/2  or  E  1/2,  you  must  hand  out  to  each  soldier  the 
TM  Extract  for  either  the  M109  or  the  MHO  howitzer.  The  TMs  are  used  with 
"Perform  PMCS  on  Howitzer." 

11.  Remind  soldiers  that  they  must  use  the  TM  Extract  for  the  PMCS  tasks.  They 
are  not  to  mark  in  the  TM  Extract. 
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MONITOR  INSTRUCTIONS 
TASK  KNOWLEDGE  TESTS 

1.  Within  each  session,  use  the  same  number  of  white  and  unwhite  (Series  1 
and  2)  booklets.  Use  the  following  test  schedule  sequence: 

Test  Session  1:  A  1  (White)  or  2  (Yellow,  Blue,  or  Pink) 

B  1  (White)  or  2  (Yellow,  Blue,  or  Pink) 

C  (White  or  Yellow)  (11B  only) 

Test  Session  2:  B  1  (White)  or  2  (Yellow,  Blue  or  Pink) 

C  (White  or  Yellow)  (11B  only) 

A  1  (White)  or  2  (Yellow,  Blue,  or  Pink) 

Make  sure  that  everyone  who  has  a  white  answer  sheet  has  a  white  cover 
sheet  on  their  test  booklet  and  the  same  for  unwhite.  Do  not  let  soldiers  open 
the  test  booklets  until  you  tell  them  to. 

2.  Make  sure  each  soldier  has  a  sharpened  pencil  with  an  eraser.  Also,  have 
available  scrap  paper  for  the  soldiers  to  use  for  any  computations  required,  and 
extra  pencils. 

3.  Read  the  following  Instructions  to  the  soldiers: 

DURING  THE  NEXT  FOUR  HOURS  YOU  WILL  BE  COMPLETING  WRITTEN  TESTS  ON  TASKS  IN 
YOUR  MOS.  THE  TEST  IS  DIVIDED  INTO  2  (11B:  3)  BOOKLETS  AND  COVERS  ABOUT  30 
TASKS.  YOUR  FIRST  BOOKLET  IS  ON  YOUR  DESK.  YOU  WILL  NOTICE  THAT  THE  BOOKLET 
HAS  A  COVER  SHEET  WITH  THE  LETTER  (A  or  8)  ON  IT  FOLLOWED  BY  EITHER  THE  NUMBER 
ONE  OR  THE  NUMBER  TWO.  YOU  WILL  TAKE  A  AND  B  (11B:  A,  B,  AND  C)  BOOKLETS.  YOU 
WILL  ALSO  NOTICE  THAT  YOUR  COVER  SHEET  ON  YOUR  TEST  BOOKLET  IS  EITHER  WHITE  OR 
YELLOW.  NOW  LOOK  AT  YOUR  ANSWER  SHEET.  IF  YOUR  TEST  BOOKLET  HAS  A  WHITE  COVER 
SHEET,  THEN  YOUR  ANSWER  SHEET  MUST  ALSO  BE  WHITE.  IF  YOUR  TEST  BOOKLET  HAS  A 
(YELLOW)  COVER  SHEET  THEN  YOUR  ANSWER  SHEET  MUST  ALSO  BE  (YELLOW).  IS  THERE 
ANYONE  WHOSE  SHEETS  DO  NOT  MATCH?  (Pause-Check)  THE  FIRST  THING  I  WANT  YOU  TO 
DO  IS  TO  ENTER  YOUR  NAME,  SOCIAL  SECURITY  NUMBER,  AND  DATE  ON  THE  SIDE  OF  THE 
ANSWER  SHEET.  PRINT  YOUR  NAME,  LAST  NAME  FIRST.  (Pause-Check)  ABOVE  THE  DATE 
WRITE  IN  AM  (PM).  (Pause) 

NOW  LOOK  AT  YOUR  ANSWER  SHEET.  NOTICE  THAT  YOUR  ANSWER  SHEET  IS  DIVIDED 
INTO  BLOCKS  A  1  OR  2  ON  ONE  SIDE  AND  B  1  OR  2  (AND  D)  ON  THE  OTHER  SIDE.  MAKE 
SURE  THE  BLOCK  YOU  ARE  WORKING  IN  MATCHES  THE  LETTER  AND  NUMBER  ON  YOUR  TEST 
BOOKLET.  THE  NAMES  OF  THE  TASKS  ARE  ON  THE  ANSWER  SHEETS  TO  HELP  YOU. 

WHEN  YOU  TAKE  THE  TEST,  READ  THE  ITEM  IN  THE  TEST  BOOKLET  AND  SELECT  THE 
CORRECT  ANSWER.  THEN  GO  TO  THE  ANSWER  SHEET  AND  DRAW  A  LINE  OR  AN  X  THROUGH 
THAT  LETTER  ON  THE  ANSWER  SHEET.  DOES  EVERYONE  UNDERSTAND?  ANY  QUESTIONS? 

(Use  an  example  to  illustrate,  if  necessary)  DO  NOT,  I  REPEAT,  DO  NOT  MARK  YOUR 
ANSWERS  IN  YOUR  BOOKLET  OR  MAKE  ANY  OTHER  MARKS'” IN  YOUR  BOOKLET.  ONFSHEET  OF 
BLANK  PAPER  IS  PROVIDED  FOR  YOU  TO  MAKE  ANY  COMPUTATIONS  ON. 
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MARK  ONLY  ONE  ANSWER  FOR  EACH  QUESTION.  USE  PENCIL.  IF  YOU  WISH  TO  CHANGE 
YOUR  ANSWER,  ER/SE  YOUR  OLD  MARKS  COMPLETELY.  ANSWER  EVERY  QUESTION— THERE  IS 
NO  PENALTY  FOR  GUESSING. 

WHEN  YOU  HAVE  COMPLETED  THE  BOOKLET,  CHECK  YOUR  WORK.  LET  ME  KNOW  WHEN  YOU 
ARE  FINISHED,  AND  I  WILL  ISSUE  YOUR  SECOND  TEST  BOOKLET. 

DO  NOT  HURRY  YOUR  WORK.  MOST  OF  YOU  WILL  FINISH  LONG  BEFORE  THE  TIME  IS 
UP. 

FINALLY,  I  REMIND  YOU  THAT  YOU  MUST  DO  YOUR  OWN  WORK  AND  THERE  WILL  BE  NO 
TALKING  ALLOWED  DURING  THE  TEST.  YOU  MAY  TAKE  BREAKS  AS  YOU  NEED  TO,  BUT  DO  NOT 
LEAVE.  LEAVE  ALL  YOUR  MATERIALS  ON  YOUR  DESK.  PLEASE  DO  NOT  TALK  ABOUT  THE 
TEST  DURING  YOUR  BREAK.  ARE  THERE  ANY  QUESTIONS?  (Wait)  OPEN  YOUR  TEST 
BOOKLET  AND  BEGIN. 

4.  Monitor  during  the  test  to  Insure  that  each  soldier  is  working  within  the 
correct  block.  This  is  especially  important  if  the  group  starts  on  the  B  block. 

5.  If  any  soldier  needs  to  leave  the  immediate  break  area,  insure  he/she  has 
your  permission.  All  soldiers  should  start  the  second  booklet  by  the  end  of  the 
second  hour.  When  a  soldier  is  finished  with  all  booklets,  he/she  may  leave. 

6.  Some  soldiers  may  ask  questions  concerning  process  or  test  content.  On 
questions  concerning  the  process,  i.e.  questions  concerning  purpose  of  the  test, 
time  limits,  test  procedures  or  possible  errors  in  the  test  (such  as  spelling), 
answer  the  question  for  the  individual  and  then  state  the  question  for  the  group 
and  answer  it  for  everyone.  These  questions  should  be  asked/answered  only 
before  the  testing  begins  or  when  everyone  is  done. 

Questions  concerning  test  content  should  be  answered  if  they  are  very 
general  questions,  i.e.  general  vocabulary  questions  or  sentence  structure. 

Every  effort  should  be  made  to  avoid  prolonged  questioning  and  "giving 
away"  the  answer.  Do  not  attempt  to  answer  technical  questions.  Respond 
by  saying  something  like:  "That  is  a  technical  question  and  part  of  the 
area  you  are  being  tested  on.  I  cannot  help  you.  Just  do  your  best." 

7.  As  much  as  possible,  keep  track  of  time  to  complete  each  booklet. 


L.2  (Cont'd.)  Monitor  Instructions  (Batch  B) 


L-6 


KNOWLEDGE  TESTS 
MOS-SPECIFIC  INSTRUCTIONS 


63B 

1.  For  the  test  "Determine  Grid  Coordinates"  in  Booklet  B1/B2,  you  must 
hand  out  maps  and  protractors.  Collect  the  maps  and  protractors  as 
soldiers  finish  with  them.  Erase  any  marks  made  on  the  map.  If  marks 
cannot  be  erased  or  if  erasures  are  apparent,  throw  the  map  away. 

31C 

1.  One  hands-on  test  will  be  administered  first.  Instructions  are 
attached  to  the  test.  When  all  soldiers  finish,  continue  with  the 
knowledge  tests. 

2.  For  the  test  "Determine  Grid  Coordinates"  in  Booklet  A1/A2,  you  must 
hand  out  maps  and  protractors.  Collect  the  maps  and  protractors  as 
soldiers  finish  with  them.  Erase  any  marks  made  on  the  map.  If  marks 
cannot  be  erased  or  if  erasures  are  apparent,  throw  the  map  away. 

3.  When  testing  Booklet  A1/A2,  you  must  hand  out  the  TM  extract  on 
maintaining  the  M17  mask. 

19E 

1.  For  the  test  "Determine  Grid  Coordinates"  in  Booklet  A1/A2,  you  must 
hand  out  maps  and  protractors.  Collect  the  maps  and  protractors  as 
soldiers  finish  with  them.  Erase  any  marks  made  on  the  map.  If  marks 
cannot  be  erased  or  if  erasures  are  apparent,  throw  the  map  away. 

2.  For  the  test  "Perform  Gunner's  Prepare  to  Fire  Checks"  in  Booklet 
BI/B2,  you  must  hand  out  the  M60A3  TM  extract. 

91A 

1.  One  hands-on  test  will  be  administered  first.  Instructions  attached  to 
the  test.  When  all  soldiers  finish,  continue  with  the  knowledge  tests. 

2.  For  the  test  "Determine  Grid  Coordinates"  in  Booklet  A1/A2,  you  must 
hand  out  maps  and  protractors.  Collect  the  maps  and  protractors  as 
soldiers  finish  with  them.  Erase  any  marks  made  on  the  map.  If  marks 
cannot  be  erased  or  if  erasures  are  apparent,  throw  the  map  away. 


L.2  (Cont'd.)  Monitor  Instructions  (Batch  B) 
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1.  When  testing  "Navigate  Point  to  Point",  in  Booklet  Al/2  you  must  hand 
out  a  protractor  and  map  extract  to  each  soldier.  Collect  the 
protractors  and  maps  as  soldiers  finish  that  task  test.  If  there  are 
any  marks  on  the  map,  throw  the  map  away. 

2.  Test  Booklet  Bl/2,  Task  "Operate  As  A  Station  In  A  Radio  Net,"  requires 
the  CEOI  extract  included  with  the  test.  Remind  the  soldiers  that  they 
must  use  the  extract  to  answer  parts  of  the  test. 

3.  When  testing  Booklet  C,  you  must  hand  out  the  M151A1  1/4  ton  truck  TM 
extract  and  the  M113A1  carrier  extract. 


L.2  (Cont'd.)  Monitor  Instructions  (Batch  B) 
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APPENDIX  M 


Instructions  For  Hands-On  Test  Suitability  Ratings 
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SUITABILITY  RATING  FOR  HANDS-ON  TESTS 


The  tasks  that  were  tested  hands-on  are  listed  on  the  next  pages.  For  each  MOS 
where  you  assisted  in  scorer-training  or  test  administration,  please  rate  the 
tasks  on  the  following  three  dimensions: 

Standardization  of  Conditions 

0  *  Cannot  standardize  relevant  conditions  (environment  or  equipment), 
within  or  among  test  sites 

1  »  Some  conditions  can  be  standardized 

2  *  Conditions  easily  reproduced  at  each  site  by  local  resources 
Standardization  of  Scorers/Scoring 

0  -  Scorers  cannot  be  trained  to  score  reliably;  much  scorer  judgment  or 
scorer/sol dier  Interaction  required;  very  difficult  to  observe,  measure 
or  record  performance 

1  ■  Scorer  reliability  is  usually  trainable 

2  *  Scorers  are  capable,  consistent  and  reliable;  performance  is  easily 

observed,  measured  and  recorded 


Task/Test  Relevance  (Fidelity 


0  »  Hands-on  test  does  not  resemble  corresponding  task;  test  performance 
greatly  removed  from  or  reduced  from  task  performance 


1  ■  Hands-on  test  is  somewhat  close  to  task  performance,  but  noticeably  not 
identical 


2  *  Hands-on  test  closely  resembles  task;  test  performance  and  task 
performance  nearly  identical 

For  tasks  that  you  did  not  observe  more  than  once  or  twice,  please  mark  "NA". 
Additionally,  if  you  do  not  know  how  the  task  is  performed,  please  mark  "NA", 
for  relevance. 


Figure  N- 
Figure  N- 
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Procedure  For  Adjusting  Knowledge  Test  Sets 


1.  Assigning  Weights  to  Hands-On  and  Knowledge  Task  Tests 

2.  Steps  for  Using  Weights  to  Reduce  Knowledge  Tests 


N-l 


This  appendix  elaborates  on  the  procedure  summarized  in  the  main  body  of 


the  companion  report.  Development  and  Field  Test  of  Task-Based  MOS-Specific 
Criterion  Measures  (Campbell,  Campbell,  Rumsey,  &  Edwards,  1985,  pp.  44-45), 
for  reducing  4-hour  knowledge  tests  to  the  2-hour  time  block  made  available 
during  Concurrent  Validation  testing.  Following  the  initial  steps  described 
on  page  44  of  that  report,  the  next  step  was  to  generate  weights  to  be  used 
in  determining  which  test  mode  was  best  measuring  each  element  of  each  task. 
These  "weights,"  which  simply  represent  integer  score  values,  were  assigned 
to  both  hands-on  and  knowledge  task  tests.  Weights  were  assigned  to  hands-on 
tests  on  the  basis  of  the  following  criteria: 

(1)  Certainty/Uncertainty.  If  the  task  test  was  not  being  revised,  or 
the  revised  test  would  have  a  tryout,  the  test  was  given  a  positive  score  of 
one.  If  the  task  test  was  being  revised  and  would  not  have  another  tryout, 
it  was  given  a  negative  score  of  one. 

(2)  Difficulty  level.  If  the  mean  score  on  a  task  was-  at  least  equal 
to  15%,  and  not  greater  than  85%,  the  task  was  given  a  positive  "weight" 
(score)  of  one.  If  the  mean  score  was  between  10  and  15%,  or  between  85  and 
90%,  the  task  test  received  a  "weight"  of  zero.  If  the  mean  score  was  less 
than  10%  or  greater  than  90%,  the  weight  given  was  a  negative  value  of  one. 

(3)  Discriminabi 1 ity.  If  the  standard  deviation  was  equal  to,  or 
greater  than,  .15  on  a  scale  of  0  to  1.0  (scores  were  recorded  as  per¬ 
centages),  the  test  was  assigned  a  positive  weight  of  one.  If  the  standard 
deviation  was  less  than  .15  but  .10  or  greater,  the  test  was  assigned  a 
weight  of  zero.  If  the  standard  deviation  was  less  than  .10,  the  test  was 
assigned  a  negative  weight  of  one. 
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(4)  Reliability.  If  the  split  half  reliability  of  the  test  was  .70  or 
greater,  the  test  was  assigned  a  positive  weight  of  one.  If  the  reliability 
was  less  than  .70,  the  test  was  assigned  a  weight  of  zero. 

(5)  Suitability.  Suitability  of  tasks  for  hands-on  testing  was 
assigned  using  procedures  described  in  the  body  of  the  report  and  in  the 
instructions  shown  in  Appendix  M  of  this  volume.  If  the  suitability  score 
was  5  or  6,  the  hands-on  test  was  given  a  positive  weight  of  one.  If  the 
suitability  score  was  4,  a  zero  weight  was  assigned.  Finally,  if  the  suit¬ 
ability  score  was  3,  a  negative  weight  of  one  was  assigned. 

Weights  were  assigned  to  written  task  tests  on  the  basis  of  the  follow¬ 
ing  criteria: 

(1)  Certainty/Uncertainty.  If  the  task  test  was  not  being  revised,  or 
the  revised  test  would  have  a  tryout,  the  test  was  given  a  positive  weight  of 
one.  If  the  task  test  was  being  revised  and  could  not  have  another  tryout, 
it  was  given  a  negative  weight  of  one. 

(2)  Difficulty  level.  If  the  mean  score  on  a  task  was  at  least  equal 
to  15%,  and  not  greater  than  85%,  the  task  was  given  a  positive  weight  of 
one.  If  the  mean  score  was  between  10  and  15%,  or  between  85  and  90%,  the 
task  test  received  a  weight  of  zero.  If  the  mean  score  was  less  than  10%  or 
greater  than  S0%,  the  task  test  was  given  a  negative  weight  of  one. 

(3)  Discriminabi 1 ity.  If  the  standard  deviation  was  equal  to,  or 
greater  than,  .15,  the  test  was  assigned  a  positive  weight  of  one.  If  the 
standard  deviation  was  less  than  .15  but  .10  or  greater,  the  test  was 
assigned  a  weight  of  zero.  If  the  standard  deviation  was  less  than  .10,  the 
test  was  assigned  a  negative  weight  of  one. 


(4)  Reliability.  If  the  coefficient  alpha  of  the  test,  as  adminis¬ 
tered  for  this  particular  MOS,  was  equal  to,  or  greater  than,  .40,  the  test 
received  a  positive  weight  of  one.  If  the  coefficient  alpha  of  the  tests,  as 
administered  for  the  MOS,  was  less  than  .40,  but  the  test  had  typically 
received  coefficient  values  of  .40  or  higher  when  administered  for  other  MOS, 
the  test  received  a  weight  of  zero.  If  the  coefficient  alpha  of  the  test,  as 
administered  for  this  MOS,  was  .40,  and  was  not  typically  higher  when  the 
test  was  administered  for  other  MOS,  or  if  the  test  had  not  been  administered 
for  other  MOS,  the  test  received  a  negative  weight  of  one. 

These  criteria  are  summarized  in  Figure  N-l.  After  the  weights  for 
both  hands-on  and  knowledge  tests  were  determined  on  the  basis  of  these 
criteria,  they  were  combined  and  used  as  shown  in  Figure  N-2. 
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Feature  -  Hands-On 


Definition 


Weight 


(1)  Certainty/Uncertainty 

(2)  Difficulty  Level 

(3)  Discriminability 

(4)  Reliability 

(5)  Suitability 

Feature  -  Knowledge  Tests 

(1)  Certainty/Uncertainty 

(2)  Difficulty  Level 

(3)  Discriminability 

(4)  Reliability 


Task  test  was  not  being  revised,  or  the 
revised  test  would  have  a  tryout. 

Task  test  was  being  revised  and  would  not 
have  another  tryout. 

Task  test  was  [not]  too  easy  or  too 
difficult: 

P  >  15*  and  P  <  85* 

P  <  15*  or  P  >  85* 

P  <  10*  or  P  >  90* 

Task  test  standard  deviation  [not]  too  low: 
S.D.  .>  15* 

S.D.  <  15* 

S.D.  <  10* 

Task  test  internal  consistency  (split  half) 
>  .700 
7  .700 

Task  test  suitability  ratings  across  three 
factors  (Standardization  of  conditions  and 
of  scorers.  Task/ test  relevance)  [not]  too 
low: 

Suitability  ■  6  or  5 
Suitability  *  4 
Suitability  ■  3 


Task  test  was  not  being  revised,  or  the 
revised  test  would  have  a  tryout. 

Task  test  was  being  revised  and  would  not 
have  another  tryout. 

Task  test  was  [not]  toe  easy  or  too 
difficult: 

P  >  15*  and  P  <_  85* 

P  <  15*  or  P  >  85* 

Task  test  standard  deviation  [not]  too  low: 
S.D.  >  15* 

S.D.  <  15* 

S.D.  <  10* 

Task  test  internal  consistency  [not]  too 
low: 

Alpha  _>  .400  for  this  tryout. 

Alpha  <  .400  for  this  MOS  tryout,  but 
higher  for  other  tryouts. 

Alpha  <  .400  for  most  tryouts  of  task 
test,  or  if  task  test  had  only  one 
tryout. 


1 

-1 


1 

0 

1 


1 

0 

1 


1 

0 


1 

0 

-1 


1 


-1 


1 

-1 


1 

0 

-1 


1 

0 

-1 


Figure  N-l.  Assigning  Weights  to  Hands-On  and  Knowledge  Task  Tests 
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STEPS  FOR  USING  WEIGHTS  TO  REDUCE  KNOWLEDGE  TESTS 


1.  Sum  the  weights  for  each  hands-on  task  test  and  each  knowledge  task  test. 
Each  hands-on  test  will  sun  to  between  -4  and  5;  each  knowledge  test  will 
sum  to  between  -4  and  4.  (In  practice,  they  both  top  out  at  about  one  or 
two.)  Any  test  with  sunned  weights  below  -1  is  by  definition  weak,  for 
purposes  of  the  final  adjustment  steps. 

2.  Compare  revised  hands-on  and  knowledge  tests  of  each  task  to  Identify 
points  of  content  overlap.  For  each  task,  note  the  number  of  knowledge 
itens  not  covered  by  the  hands-on  tests. 

3.  Reduce  the  knowledge  tests  by  following  the  steps  in  below.  After  each 
step,  check  the  number  of  knowledge  items.  Continue  the  process  until  that 
number  is  602  -  752  of  the  original  (field  test)  set  (depending  on  the 
MOS).  When  the  set  of  knowledge  items  is  sufficiently  reduced  from  the 
number  field  tested,  stop.  Finalize  tests  for  proponent  review. 


Steps  for  Reducing  the  Knowledge  Test  Set 


Step 

If  HO  is: 

And  K  is: 

And  r  is: 

Then 

the  Rule  is: 

la 

weak 

good 

Test 

K  and  HO 

lb 

none 

any 

— 

Test 

K 

2 

good 

weak 

— 

Test 

K'  and  HO 

3 

weak 

weak 

— 

Test 

K'  and  HO 

4 

good 

good 

high 

Test 

K'  and  HO 

5 

good 

weak 

Test 

HO  (drop  K* ) 

6 

good 

good 

low 

Test 

K*  and  HO 

7 

good 

good 

high 

Test 

HO  (drop  K' ) 

NOTE: 

HO  is  a  hands-on 

test  field 

-tested  in  any  MOS. 

K  is  a  knowledge  test. 

K'  is  a  knowledge  test  reduced  for  hands-on  overlap, 
r  is  hands-on  correlation  with  knowledge. 

Count  items  after  each  step;  stop  when  number  of  items 
is  reduced  to  target  number. 
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Figure  N-2. 


Steps  for  Using  Weights  to  Reduce  Knowledge  Tests 
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APPENDIX  0 

Distributions  of  Knowledge  Items  on  Difficulty 
and  Item-Total  Correlation  in  Nine  MOS 
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DISTRIBUTIONS  OF  KNOWLEDGE  TEST  ITEMS 
ON  DIFFICULTY  AND  ITEM-TOTAL  CORRELATION 


NNON  CREWMAN 
DIFFICULTY  I -.19-0.0 


ITEM- TOTAL  CORRELATION 
01-.20  .21-. 40  .41- .60  .61-.80 


.81-1.00 


TOTAL 


-  MOTOR  TRANSPORT  OPERATOR 

ITEM-TOTAL  CORRELATION 

DIFFICULTY  -.19-0.0  .01- .20  .21-. 40  .41-. 60  .61-. 80 


TOTAL 


9 

16 

120 

45 

46 


236 


.81-1.00 


TOTAL 


NISTRATIVE  SPECIALIST 


ITEM-TOTAL  CORRELATION 

DIFFICULTY  -.39— .20  -.19-0.0  .01-. 20  .21- .40  .41-. 60 


.61- .80 


TOTAL 


0 

2 

8 

1 

2 

4 

3 

8 

0 

0 

1 

12 

44 

9 

4 

0 

4 

28 

5 

4 

0 

2 

23 

2 

3 

5 

23 

111 

17 

13 

TOTAL 


0-2 


DIFFICULTY 


.01  -  .20 
.21  -  .40 
.41  -  .60 
.61  -  .80 
.81  -  1.00 


TOTAL 


MILITARY  POLICE 
ULTY  1  -.19-0.0 


ITEM-TOTAL  CORRELATION 


01-.20 


21-.40 


41-.60 


.61- .80 


TOTAL 


10 

14 

47 

25 

81 


177 


DIFFICULTY 


.01  -  .20 
.21  -  .40 
.41  -  .60 
.61  -  .80 
.81  -  1.00 


TOTAL 


.39— .20 


ITEM-TOTAL  CORRELATION 
19-0.0  .01-. 20  .21- .40  .41-. 60 


61-  .80  .81-1.00  TOTAL 


61- .80  .81-1.00  TOTAL 


DIFFICULTY 


.01  -  .20 
.21  -  .40 
.41  -  .60 
.61  -  .80 
.81  -  1.00 


TOTAL 


-.19-0.0 

ITEM- TOTAL  CORRELATION 
.01- .20  .21- .40 

41- .60 

.61- .80 

2 

9 

4 

0 

0 

5 

6 

10 

1 

0 

0 

13 

54 

3 

10 

2 

2 

26 

4 

6 

1 

2 

29 

11 

11 

10 

32 

123 

19 

27 

TOTAL 


15 

22 

80 

40 

54 


211 


ITEM- TOTAL  CORRELATION 

DIFFICULTY  -.19-0.0  .01-. 20  .21- .40  .41-. 60  .61- .80  .81-1.00  TOTAL 


DIFFICULTY 

-.39— .20 

-.19-0 

ITEM- TOTAL  CORRELATION 
.0  .01-. 20  .21-  .40 

o 

VO 

• 

1 

r—4 

O 

CO 

1 

H 

VO 

• 

TOTAL 

.01  - 

.20 

2 

1 

2 

3 

0 

0 

8 

.21  - 

.40 

2 

4 

9 

0 

0 

15 

.41  - 

.60 

3 

12 

51 

3 

4 

73 

.61  - 

.80 

2 

5 

32 

6 

4 

49 

.81  - 

1.00 

1 

8 

49 

17 

16 

91 

TOTAL 

1 

5 

23 

111 

17 

13 _ 

170 

0-4 


